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5 SYSTEM AND METHOD FOR TARGETING OBJECT ORIENTED 

AUDIO VIDEO CONTENT TO USERS 



Field of the Invention 

10 The present invention relates to electronic information access systems and more 

specifically to a media object based system and method for targeting information to users based 
upon user profiles. 

Background of the Invention 

15 Television, radio, Internet, and other multimedia content creators and transmitters often 

rely upon mass market commercial advertising to sponsor programming presented to their 
viewers and listeners (for purposes of simplicity, those who receive programming in audio, video, 
textual, graphic, or other formats shall be referred to as "users")- Commonly, such advertising is 
provided as distinct audio and video segments which are presented during an interruption in a 

20 program the user has selected (for example, a Nike® commercial during a football game, a banner 
advertisement on an Internet page, or a logo on a video screen) . While currently used advertising 
mechanisms have been marginally successful in promoting goods and services to non- 
participatory audiences (i.e., users who do not interact with the program and merely watch, listen, 
or read the presented content), such mechanisms leave a lot to be desired in today's multimedia, 

25 interactive environment. More specifically, such advertising mechanisms are often undesirable 
due to bandwidth considerations, transmission formats, and the fact that they do not facilitate the 
targeting of advertising and/or programming to users. As a result, much of the advertising and 
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programming content currently provided is often misdirected, presented to a disinterested 
audience, or completely ignored by users. 

Further compounding the task of garnering user interest in advertising and programming 
segments, is the fact that most of the programming currently being offered is provided on a 
5 national and/or regional basis. As a result many of the advertisements are also national/regional 
in scope and are not targeted to specific demographic groups or users matching specific user 
profiles. Often during the national/regional broadcasts, local affiliates attempt to focus their 
advertising slots and programming to the local audience by inserting locally generated 
commercials into the program, providing scrolling messages on the bottom of video displays (for 

10 example, severe weather warnings), and by other techniques. However, even the local affiliates 
are not specifically targeting advertising or programming to specific users. In short, the systems 
and processes currently available do not allow the advertisers, content creators (for both the 
advertisement and the underlying program), and/or transmitters to specifically tailor and target 
advertising and programming to specific users. As a result, much of today's commercial 

15 advertising and programming is ineffective, mis-targeted, and often totally ignored by users. 

Various approaches have recently been proposed to address the inefficient use and 
targeting of advertising/programming, especially television and video based advertising. 
Commonly such approaches provide for the separation of a program (for example, a football 
game) into numerous 6 MHz analog channels, upon which unique commercial advertisements are 

20 inserted, with each channel providing different commercials. The user's system receives all of 
these channels and, according to user selections or a pre-set user profile, selects one of the 
channels for presenting the advertising and associated programming. As is readily appreciated by 
those skilled in the art, such an approach generally provides a limited number of programming 
signals into which the targeted advertising may be inserted (thus, minimizing the specificity of 

25 which such targeted advertising may be provided), while also requiring multiple signals to 



transmit the same underlying program and thus wasting precious bandwidth and transmission 
resources. 

Additionally, those systems which attempt to target advertising to users generally rely 
upon user responses to questionnaires and other survey information and do not provide 
5 advertisers with sufficient, real-time information to tailor advertising to specific users. As such, a 
need exists for systems and processes, which provide efficient and effective targeted 
advertising/programming to users without requiring massive tradeoffs in bandwidth, transmission 
capacities, or other variables. 

10 Summary of the Invention 

The present invention provides a system and process for targeting programming 
(including, but not limited to, advertising) to users by utilizing media object to provide specific 
content to a user, or groups of users, based upon user profiling information. More specifically, 
the present invention provides a system and method for generating media object based content, 

15 formatting and transmitting the content, and receiving and presenting the content to a user. The 
media object based content is preferably presented to users based upon profile established for the 
user. As such, the systems and processes of the present invention enables media object content 
creators, producers, broadcasters, and others involved in the transmission of multimedia content 
to users to custom tailor such content for unique users and/or groups of users based upon user 

20 profiles. 

The system utilizes user purchase behaviors, on-line activity, responses by a user to a 
survey, demographic information, user viewing habits, statistical information, regional 
information and other information to generate a user profile. The user profile may be generated 
at a receiving system, a transmitting system, or any other location (for example, by an internet 
25 service provider). Such profile information is utilized to identify user' s based upon user profiles, 



create media objects for such profiles and present specific media objects to specific user profiles. 
As such, the present invention facilitates targeted advertising, targeted programming and other 
presentations to users. Instead of requiring all users to see a specific program, advertisement, 
game, show, or other presentation, the present invention facilitates the customization of each 
5 user's program based upon user profiles and the interchanging of one media object for another 
media object. 

More specifically, in the preferred embodiment of the present invention, a transmitting 
system, a receiving system and a user profiling system are used to target content to users. The 
content is provided in the form of media objects, preferably using the MPEG-4 format, however, 

10 other formats may also be utilized, created by media object creators on a pre-recorded or live 
basis. The media objects are suitably communicated to the receiving system via any transmission 
medium, including broadcast, wireless, wireline, satellite, cable, fiber optics, microwave, and 
millimeter wave. Similarly, network connections may also be utilized to transmit the user profile 
information and/or the media objects to the user. Such network connections may utilize any 

15 known in the art including, the Internet, intranet, ATM network, private network, wireless 
network, wide area network, local area network, and a public network. 

Further, user-profiling information is utilized by the receiving system and/or the 
transmitting system to determine which of a plurality of media objects to present to a specific 
user or group of users at any instance during a program, advertisement or other multi-media 

20 presentation. The media obj ects may be provided in any medium desired including, for example, 
audio, video, graphical, and textual. Further, such media objects may be provided by remote 
creators, via local storage mediums, via network broadcasts, cable systems, and other mediums to 
the user's receiving system. Similarly, the user profiling information may be local to the user or 
remote, for example, accessible by the media object creators and/or the transmitting system. 
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Further features and functions of the present invention will become apparent from a 
consideration of the following detailed description, drawing figures and claims. 

Brief Description of the Drawing Figures 

5 Figure 1 is a schematic representation of a preferred embodiment of a system providing 

media object based content targeted to users based upon user profiles. 

Figure 2 is a schematic representation of the receiving system of Figure 1 by which the 
media objects are received and selected for presentation to a user by the preferred embodiment. 
Figure 3 is an illustrative example of the receiving system of Figure 2 for selecting one 
10 media object from a plurality of media objects provided in an automobile advertisement. 

Figure 4 is a flow chart illustrating the process by which the present invention may be 
utilized to push targeted advertising and programming to users via media objects selected based 
upon a user profile. 

Figure 5 is a flow chart illustrating the process by which the present invention may be 
15 utilized by a user to pull media objects, which are provided to the user based upon their user 
profile. 

Detailed Description 

The present invention provides a system and method for targeting audio and video content 
20 (for example, advertising and programming) to users by utilizing media objects and user 
responses thereto. Those skilled in the art of data transmission techniques and capabilities are 
familiar with the MPEG-4 (Motion Picture Expert Group) standard which has been promulgated 
in order to standardize the creation, transmission, distribution, and reception of media objects 
based upon audio, video, graphical and various other forms of data and information (hereafter, 
25 generically referred to as "content"). The present invention preferably utilizes the media object 
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specifications of MPEG-4 and various other object based data transmission systems to generate 
user profiles, transmit content to users based upon such profiles, and thereby provide targeted 
advertising/programming, in the form of media objects, to such users. While the present 
invention is herein described with reference to the MPEG-4 standards, it is to be appreciated that 
5 various other systems and methods for transmitting and receiving media object based content 
may be utilized in conjunction with the present invention. Such systems include, for example, 
the Sony PlayStation®, which provides the capability of rendering graphical media objects over 
real-time video. 

Additionally, throughout this specification, reference is made to media object oriented 

10 content, MPEG-4, and various other terms and standards promulgated by the MPEG. As used 
herein, "media objects" are defined in accordance with the definitions and descriptions provided 
in the "Overview of the MPEG-4 Standard" provided by the International Organization for 
Standardization, ISO/IEC JTC 1/SC29/WG11 N3156, December 1999/Maui, the contents of 
which are herein incorporated by reference. More specifically, "media objects" are commonly 

15 representations of aural, visual or audio-visual content which may be of natural or synthetic 
origin (i.e., a recording or a computer generated object). As is commonly known in the art, such 
media objects are commonly organized in a hierarchy with primitive objects (for example, still 
images, video objects, audio objects, etc.) and coded representations of objects (for example, text, 
graphics, synthetic heads, and synthetic sounds). These various objects are utilized to describe 

20 how the object is utilized in an audio, video, or audio-visual stream of data and allow each object 
to be represented independently of any other object and/or in reference to other objects. For 
example, a television commercial for an automobile may consist of an automobile, a scenery or 
route upon which the automobile travels, and an audio signal (for example, a voice describing the 
characteristics of the automobile, background sounds adding additional realism to the 

25 presentation, and background music). Each of these objects may be interchanged with another 
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object (for example, a car for a truck, or a rock soundtrack for an easy listening soundtrack) 
without affecting the other objects, or specifically affecting the other objects - as desired by the 
content creator. 

Additionally, throughout this application a "stream of data" (a.k.a., "streaming" or 
5 "streaming media") is herein defined in accordance with the definition provided by Webopedia 
(www.webopedia.com ) as "a technique for transferring data such that it can be processed as a 
steady and continuous stream ... [such that] a client browser or plug-in can start displaying the 
data before the entire file has been transmitted." Examples of technologies capable of supporting 
"streaming" include, but are not limited to, Progressive Network's RealAudio®, Real Network's 

10 Real Video®, Microsoft's NetShow®, and RTSP (Real Time Streaming Protocol). The present 
invention may utilize any of the various streaming technologies currently, or in the future, 
available as desired and/or as necessary to transmit media objects. Additionally, it is to be 
appreciated that non-streaming technologies may also be utilized in conjunction with the present 
invention including, for example, content presented via a compact disc or digital video disc, by 

15 completing a file transfer before beginning the presentation, or other transfer means. 

Further, it is to be appreciated that the present invention may be utilized in a multicast 
environment, i.e., where the content/media objects are transmitted to a select group of users. 
Similarly, the present invention may be utilized in a unicast environment (i.e., where the 
content/media objects are transmitted to a single user) and/or a broadcast environment (i.e., 

20 where the content/media objects are transmitted to every user connected to a network, or, in the 
case of a radio frequency or satellite broadcast, to all users equipped with a proper receiving 
system). 

As shown in Figure 1 , one system 1 00 implementing the present invention includes three 
major elements: a transmitting system 102, a receiving system 104, and a user profiling system 
25 1 06. Each of these three major elements is explained further in greater detail below. 



Additionally, the system 100 preferably includes media object creators 108, which create content 
in the form of media objects, which are utilized in the targeted advertising. The system 100 also 
preferably includes various ancillary devices such as user input devices 1 10, presentation systems 
112, and data storage devices 1 14. The system 100 preferably includes all of those components 
5 and elements necessary to provide media object based targeted advertising. It is to be 
appreciated, however, that such components and elements may be utilized separately, in 
conjunction with existing content distribution systems, and/or as components of other systems 
without departing from the spirit or scope of the present invention. 

Referring again to Figure 1 and as mentioned previously, one of the major elements 

10 utilized in the preferred system 100 is the transmitting system 102. The transmitting system 102 
may include any medium, system, or device capable of communicating media object oriented 
content to a user including, but not limited to: wireless transmission systems (for example, 
analog and digital television broadcasts, analog and digital radio broadcasts, analog and digital 
wireless communications networks, microwave systems, millimeter wave systems, infrared 

15 systems, and satellite broadcast systems); wire based transmission systems (for example, analog 
and digital cable systems, coaxial connections, fiber optic links, telephone systems, closed circuit 
systems, Very high speed Digital Subscriber Lines (VDSL), VGD, and any other wired 
connection); stand-alone systems for providing pre-recorded programs (for example, programs 
provided on compact discs, digital versatile discs, video tape, PlayStation cartridges, memory 

20 sticks, magnetic storage mediums, optical storage mediums, data storage devices, Flash memory, 
random access memory (RAM), and read only memory (ROM)); and network connections (for 
example, the Internet, private network, public network, wireless network, wired network, ATM 
networks wide area network, local area network, and intranet). Such media objects are 
communicated to the transmitting system by the media object creators 108 over an interface 105 

25 between the media object creators and the input port 103. The interface 105 may be any which 



facilitates the transfer of media objects to the transmitting system. Similarly, the input port 103 
may include any additional processor necessary to format a media object received in a given 
format into a format compatible with the transmitting system. 

Additionally, the transmitting system may transmit the media objects as a live signal or as 
5 a pre-recorded signal from a previously recorded medium, and with respect to an Internet 
connection, the media obj ects may be streamed. Similarly, media objects may be communicated 
to users using any of the various transmission protocols known in the art including, for example, 
IEEE 802.1 IB, RTP, UDP, TCP/IP, Ethernet, and ATM. Further, the media objects may be 
formatted according to various schemes including, MPEG-1, MPEG-2, MPEG-4, MPEG-7, 

10 JPEG, motion JPEG, GIF, QuickTime, ActiveMovie, digital video interactive (DVI), Indeo, and 
IP wrapped MPEG-4 for streaming. As such, any medium, system, device, or protocol, which 
facilitates the transfer of media object oriented content to a user may be utilized by the system 
100. In short, the transmitting system 102 is content, device, system, medium and protocol 
independent, provided the selected system supports the transmission and presentation of media 

15 objects to users. 

The media object creators 108 may also utilize numerous software programs, content 
creation techniques, and content capturing devices to generate the media objects utilized in the 
system 100. As stated previously, media objects may include natural, synthetic, and combined 
natural/synthetic content. The media object creators 108 may utilize any of the various devices 

20 and methodologies commonly known to capture or create such content. For example, one creator 
may utilize microphones to capture audio sounds whereas another creator may utilize a 
synthesizer to generate the desired audio sounds. The present system 100 is capable of providing 
targeted advertising and programming to users regardless of the techniques and systems utilized 
by the media object creators 108. Further, Figure 1 identifies three types of data commonly 

25 utilized in creating media objects, namely: video data 116, audio data 118, and graphic/textual 



data 120. The system 100, however, is not limited to specific datatypes and may incorporate any 
type of data including, but not limited to, audio data, video data, graphic data, textual data, 
animations, multimedia, slow frame video data, video stills, sequences of individual frames, 
virtual reality data, live data, pre-recorded data, and computer generated data. Additionally, the 
5 media object creators may provide media object based content targeted to a user in any type of 
program including, for example, a news program, an advertisement, a sports program, an 
entertainment program, a music video, a game show program, a motion picture program, a video 
game, a video program, an audio program, an educational program, a live program, a pre- 
recorded program, and in a noncommercial program. Similarly, the media objects may relate to a 

10 polling question, provide responses by the user or the media object creator to queries, and may 
facilitate bi-directional communications (for example, via a chat interface, instant messaging 
interface, or an electronic mail interface) between and among the media object creator, the 
transmitting system, the receiving system, and the user. The system 100 suitably includes those 
software and/or hardware components necessary to provide a chat interface, instant messaging 

15 interface and/or an electronic mail interface. Such interfaces are known in the art, and any of 
such may be suitably incorporated into the system 100 as desired. Additionally, such interfaces 
may be provided between users, the transmitting system, media object creators, other users, and 
any other person or entity using a compatible system. 

As previously mentioned above, the system 100 also includes a presentation system 112. 

20 The presentation system 112 provides at least one output device by which the content of the 
media objects is presented to a user. Any device capable of presenting media objects to users 
may be utilized as the presentation system 112. Such devices include, but are not limited to, 
television receivers, home theater systems, audio systems, computer workstations, laptop 
computers, personal data assistants, set top boxes, in-home communications systems, wireless 

25 communication systems (for example, pagers and wireless telephones), web tablets, virtual reality 



systems, web phones, printers, heads-up display, gaming consoles, tactile or sensory perceptible 
signal generators (for example, a vibration or motion), and various other devices or combinations 
of devices. For example, a presentation system may include a video monitor, an audio sound 
system, and a motion generator such that a program (for example, one simulating traveling in a 
5 car or plane) is simulated using various synchronized media objects. In short, the presentation 
system 112 is not to be construed as being limited to any specific systems, devices, or 
components, and may utilize any device capable of presenting media objects to a user. 

Similarly, numerous input devices 110 may be utilized by a user to identify themselves, 
select media objects, select programming signals, and respond to queries. Such input devices 

10 include remote control devices, keyboards, scanners (for example, retinal and fingerprint), 
mouse, trackballs, virtual reality sensors, voice recognition systems, voice verification systems, 
push buttons, touch screens, joy sticks, brain wave devices, and other such devices. 

Additionally, data storage devices 114 are preferably utilized in the system 100 for the 
temporary or permanent storage of media objects, the content provided in the media objects, 

15 transmission signals (for example, in decompressed and/or demultiplexed formats), profile 
information, operating routines, and any other information utilized by the system 100. The data 
storage devices 114 may be provided in conjunction with the receiving system 104, may be a 
stand-alone device co-located with the receiving system 104, may be remotely accessed (for 
example, via an Internet connection), may be provided in the transmitting system 102, in the user 

20 profiling system 106, with the media object creators 108, or at any other location in the system 
100. In the preferred embodiment, the data storage devices are provided with the receiving 
system 104. The data storage devices 1 14 may also utilize a combination of local and remote 
storage devices in order to provide the desired features and functions of the system 100. Those 
skilled in the art appreciate that various data storage devices, algorithms, programs, and systems 

25 may be utilized in conjunction with the system 100. Examples of such data storage devices 



include, but are not limited to, hard drives, floppy discs, CD ROMS, DVDs, tape drives, memory 
sticks, remote databases, and local databases. 

As previously mentioned herein, the system 100 also includes a receiving system 104, 
which receives the signal provided by the transmitting system 102. As shown in Figure 2, an 
5 illustrative embodiment of a receiving system 104 is provided. It is to be appreciated that such 
system 104 may be provided in hardware and/or software formats and Figure 2 is intended to 
illustrate those functions commonly performed by an MPEG-4 equipped receiving system. The 
actual layout and configuration of such functional components may vary as desired and 
commonly appreciated by those skilled in the art. The system 104 is preferably configured to 

10 support the reception, selection, and composition of media object based content into audio, video, 
and audio-video presentations. However, various other forms and methods of presenting content 
may also be supported by the receiving system 104 including graphical, textual, and virtual reality 
content. Additionally, for purposes of the present description, the receiving system 104 is 
described in the context of presenting an audio-video program (with or without graphics or text 

15 enhancements such as MPEG stills), but the receiving system 1 04 is not to be construed as being 
so limited. 

As shown, the receiving system 104 preferably includes a transceiver/interface 202. The 
transceiver/interface 202 provides the communications connectivity capabilities necessary to 
receive and transmit communications signals containing media objects or other information 

20 between various sources, via various mediums. More specifically, the receiving system 104 is 
not limited to receiving media objects in communications signals from only specific sources or 
only via specific mediums and may be configured to support any source and/or medium and 
provide bi-directional communications, as desired. Those skilled in the art appreciate that the 
transceiver/interface 202 may utilize various components and systems to receive and process 

25 communications signals containing media objects. For example, receiving a communications 



signal from a CD-ROM player might entail utilizing a serial bus, an RCA-plug, a parallel port, or 
even a Universal Serial Bus, RGB, firewire (JEEE1494), or any other conventional interface. 
Whereas, receiving a satellite or radio frequency broadcast may require antennas, low noise 
amplifiers, low noise block-down converters (LNBs), filters, receivers, and various other 
5 components, all of which are known in the art. As such, the receiving system 104 may be 
configured to receive communications signals containing media objects from any medium and, if 
desired, from multiple mediums simultaneously. Additionally, the receiving system 104, as 
desired, may be configured to transmit information to the transmitting system 102 and/or the user 
profiling system 106. 

10 In addition to establishing the desired communications links with the transmitting system 

102, the transceiver/interface 202 also preferably performs standard communications signal 
processing functions; for example, separating the various communications signals into separate 
channels and sub-channels, filtering data, restoring lost data, and those various other functions 
commonly performed by receiving devices. As shown in Figure 2, the transceiver/interface 202 

15 preferably breaks-out various sub-channels (for example, an audio, video, graphic, and a textual 
sub-channel) from a selected channel and provides each of the sub-channels to the appropriate 
demultiplexers 204. For example, a cable signal often contains numerous programming signals 
(i.e., ABC, CBS, FOX, etc.). Each of these programming signals contains sub-channels (e.g., 
multiple tracks) in which program elements provide video, audio, and textual information. Those 

20 skilled in the art appreciate that such programming signals and sub-channels (e.g., multiple 
tracks) may be provided in digital or analog signal formats. Additionally, the 
transceiver/interface 202 separates the incoming cable signal into the various programming 
signals and, based upon the user's selection, processes the selected programming signal(s) by 
breaking out the various program elements and sending such elements to the corresponding 

25 demultiplexers 204. The separation of transmitted signals into channels and program elements is 



well known in the art. The receiving system 104 may utilize any methodology and/or 
components which provide/support the features and functions specified herein and those features 
and functions commonly performed by receiving systems. 

Upon receiving a breakout segment of the program elements from the received 
transmission signal, the demultiplexers 204 preferably select specific data packets (which are 
preferably specific media objects) from a plurality of available data packets. Next, the selected 
data packets are either provided to the corresponding decompressor 208 or sent to a buffer 206 in 
which the selected data packet is temporarily stored until needed. As is commonly known, 
communications signals are commonly transmitted after being compressed utilizing a data 
compression algorithm. The use of data compression and decompression algorithms is well 
known in the art, and is not discussed in detail herein. Similarly, the use of buffers to store data 
packets for either real-time streaming or later playback is well known. The system 100 may 
utilize buffers of various sizes and configurations, as necessary, to support the various data types 
and data sizes being received by the receiving system 104. 

When the time for presenting a specific media object arrives, the data packet containing 
the media object has preferably been retrieved from the buffer 206, as necessary, and 
decompressed 208. The decompressed media object is provided to the composition generator 
210. The composition generator 210 composes the various data packets into a composite 
presentation signal. The composite presentation signal may include video elements, audio 
elements, textual elements, graphical elements, virtual reality elements, and other elements. 
These elements may then be presented to a user via various devices including, but not limited to, 
television receivers, home theater systems, audio systems, computer workstations, laptop 
computers, personal data assistants, set top boxes, in-home communications systems, wireless 
communication systems (for example, pagers and wireless telephones), web tablets, virtual reality 
systems, web phones, printers, heads-up display, gaming consoles, tactile or sensory perceptible 

14 



signal generators (for example, a vibration or motion), and various other devices or combinations 
of devices. The system 100 supports all levels of interactive multi-media presentation systems 
and the providing of media objects to users of such systems. 

Coordination and control of the various before mentioned functions of the receiving 
5 system 104 are provided by the controller 212. The controller 212 provides the various control 
signals needed to control the numerous components (i.e., the transceiver/interface, 
demultiplexers, buffers, decompressors, and composition generator) utilized by the receiving 
system 104 to present media object based content targeted to users. The controller 212 also 
contains a timing and synchronization unit, which preferably coordinates and synchronizes the 

10 spatial and temporal presentation of the various media objects provided in a program. For 
example, an audio-visual program might contain a synthetic talking head. The timing and 
synchronization unit coordinates the presentation of the synthetic talking head and the 
corresponding audio media objects such that the head appears to actually speak the words 
provided in the audio media objects. Additionally, the controller 212 is preferably in 

15 communication with a user device 1 10. The user device 1 10 facilitates interaction between the 
user and the receiving system 1 04 by receiving selections, requests, queries, and responses by the 
user to the system 100. 

The controller 212 is also, preferably, in communication with a data storage device 1 14. 
As mentioned previously, the data storage device 114 may be remote or local to the receiving 

20 system 104, as desired. The controller 212 preferably retrieves various configuration and control 
information from the data storage device 1 14, as necessary, to receive, process and present the 
media objects. The data storage device 1 14 may also be utilized to store user profile information, 
parental control information, provide additional buffer space for media objects, and perform 
various other commonly provided data storage functions. 
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In addition to providing transmission and reception systems for media objects, the present 
invention provides for the targeting of such media objects to users based upon user profiles and 
other demographic information. Referring again to Figure 1 and as mentioned previously, the 
system 100 includes a user profiling system 106. The user profiling system 106 utilizes 
5 demographic and/or user specific information to determine which of the various media objects 
are available, at any time, to present and target to a user at aparticular instance during a program. 
The user profiling system 106 uses information collected via a variety of sources and/or 
methodologies to develop a user profile and determine which media objects to present to a user 
during a program. In a most basic embodiment, the user profile information is based upon 

10 demographic or regional information collected/generated by the transmitting system 102 or by the 
media object creators 108. The user profiling system 106 may also be provided by an online 
service provider who generates user profiles based upon user responses to surveys, web page hits, 
demographic information, user viewing habits (for example, compilations thereof provided by a 
cable operator or via a TiVO® unit), purchasing behavior (both on-line and non-on-line 

15 purchases), regional information, and/or statistical information. 

For example, automobile companies realize that residents of San Diego generally 
purchase more convertibles than residents of Seattle. As such, a national television 
advertisement for a Ford Mustang® might include a media object directing a San Diego network 
affiliate's transmitting system 102 to select from a national data feed media objects (i.e., a data 

20 packet) depicting a convertible in San Diego. Meanwhile, during the broadcast of the same 
commercial, the network broadcaster may direct the Seattle affiliate to select media objects 
depicting a hard top Mustang in Seattle. The various other elements of the commercial (i.e., the 
actors, voice, and music) might be the same nation-wide with only the media objects for the car's 
roof and the baseline scenery being interchangeable. 
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Similarly, the selection of the media object (i.e., the convertible or the hard top roof) 
might also be accomplished by the receiving system 104, with or without user input. In this 
embodiment, both media objects (the convertible roof and the hard top roof) are transmitted as 
multiplexed data packets in the same programming signal to the receiving system 104. The 
5 receiving system 1 04 accesses a geographic region indicator which is preferably stored in the data 
storage device 1 14 to determine which of the received data packets the demultiplexer 204 is to 
select and provide to the other devices within the system 104 for further processing. As may be 
appreciated, the data storage device 1 14 may be automatically or manually set with a region 
indicator via commonly known processes such as using a telephone area code to designate a 
10 geographic area. Those skilled in the art appreciate the various methods by which a receiving 
system 1 04 may be configured to respond to a regional, demographic, or other generic (non-user 
specific) indicator. 

In another embodiment, the system 1 00 is configured to provide even more specific media 
object based targeted content. In this embodiment, the system 100 is configured to utilize 

15 specific information provided by the user, for example, answers to a survey or questionnaire. 
Based upon the user provided information, the user profiling system 106, which is preferably co- 
located with the transmitting system 102 and/or the media object creators 108, collects the data, 
filters the data, sorts the data, and manipulates the data to generate a user profile. Statistical 
sampling and other data manipulation techniques may also be utilized to characterize and 

20 establish user profiles for specific users or other groupings of users (for example, a grouping 
based upon geographic location). Additionally, V-chip data (i.e., a chip utilized to set parental 
controls on a multimedia device such as a television) may be utilized to generate a user profile. 
For example, information provided by a parent with four children aged 6 months to 10 years old 
may be used to generate a profile of an individual who might be interested in purchasing a mini- 

25 van. In contrast, a single professional with no children would probably generate a profile of a 



user interested in a sports car. Thus, the system 100 uses the provided information and/or other 
information and statistics to generate a user profile. The user profile information, in turn, is 
utilized by media object creators 108, broadcasters, local affiliates, and others on the transmitting 
side of the system to provide targeted advertising and programming to users by originating and 
5 transmitting media objects targeted to specific users and/or classifications of users. 

Throughout this description reference has been made to using media objects to provide 
targeted advertising and programming based upon user profiles. While the traditional methods of 
advertising goods and services (for example, commercial advertisements during interruptions in a 
program) may be utilized by the system 100, the system also supports non-interrupting 

10 advertising and programming by providing program content (as compared with the content of an 
advertisement) in which media objects in the program itself are selected and presented to the user 
based upon a user profile. These media objects may then be selected, deleted and saved by the 
user while being presented with the underlying program. Such actions by the user, for example, 
deleting a first media object while selecting a second media object, may be suitably utilized by 

15 the system to determine a user profile. For example, a user who selects a sports car might result 
in a profile of a person who is more concerned about driving excitement than the person who 
selects an economy car. Such profile information is suitably stored by the receiving system 104 
and/or the transmitting side (i.e., the transmitting system and/or the media object creators) such 
that subsequent media objects may then be inserted by the program creators and/or the 

20 transmitting system into the programming signal sent to the profiled user. In this manner, the 
program content presented to a user may be uniquely tailored to a user such that targeted content 
is provided without any user actions required and without any interruptions of the advertising 
and/or programming. 

The system 100 may also be configured to operate at a highly interactive level of user 
25 profiling. In this embodiment, the user profiling system 106 is preferably in communication with 
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both the transmitting system and/or the media object creators 108, and the receiving system 104. 
As the user selects programs and content to receive, the receiving system 104 relays such 
information to the user profiling system 106 via a communications link. The user profiling 
system 106 may be a stand-alone system, an Internet based system, or any other system capable 
of generating user profiles based upon user viewing habits (or listening habits - in the case of an 
audio program) and/or purchasing behavior. Similarly, the communications link between the 
receiving system 1 04 and the user profiling system 1 06 may be any link including, but not limited 
to, an Internet link, a telephone link, a digital subscriber link, a wireless link, and a cable link. 
Additionally, communications links between the user profiling system 106, the transmitting 
system 102, the media object creators 108, and the data storage device 1 14 are also preferably 
provided, thereby allowing the media object creators and transmitters (as desired) to program 
media object selection guidelines into the receiving system 104, at any time, based upon the user 
profile. 

While the user profiling system 106 is preferably configured to interface with both the 
transmitting system 102 and the receiving system 104, the system 100 may also be configured 
such that user profiling information and features are utilized only by the receiving system 104. In 
such an embodiment, the media object creators 108 and transmitting system 102 suitably create 
media objects for predetermined profiles and transmit such media objects to the receiving system 
104. The receiving system 104 then determines which of the various media objects to use based 
upon the user profile provided by the receiving system 104 and/or the data storage device 114. 
As such, this embodiment does not require bi-directional communication links between the 
receiving system 104 and the transmitting system 102. 

One example of such an embodiment in use is illustrated in Figure 3. As shown, the 
receiving system 1 04 receives a program containing multiple video and audio objects of which at 
least one of each is selected for presentation to the user based upon a locally stored user profile. 
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For example, an automobile manufacturer desires to market their vehicles during the Super Bowl. 
Instead of presenting a separate commercial for each of its potential markets (for example, the 
sports car, mini-van, and sport utility markets), the manufacturer merely provides a single 
commercial with multiple media objects (for example, video media objects and associated audio 
5 media objects for each type of vehicle) while also providing a baseline promotional media object 
320 (for example, a promotional special such as 2.9% financing or $1000 back) overlaid on a 
given background media object 312. Further, real time marketing conditions can dictate the 
selection and presentation of an object. For example, local sales taxes may be included in the 
sales price in one town (or geographical area) versus another town, thus leading to different 

10 promotional objects. 

Each of these media objects are received by the receiving system 104 and preferably 
broken out into video data 302 and audio data 304 streams, each of which contains numerous 
data packets representing individual media objects. For example, video media objects for a 
sports car 306, a minivan 308, and a sport utility vehicle 310 are provided as well as the 

15 corresponding audio media objects 314, 316, and 318. The controller 212 suitably determines, 
based on the user profile and/or other variables (for example, a demographic area), which of the 
media objects to select 324 and 332. 

Additionally, in certain embodiments wherein the advertiser desires a digital version of a 
user to appear in the commercial 322, such a media object may also be generated and selected. 

20 For example, the commercial might superimpose a media object of the user as the driver of the 
vehicle. Similarly, audio data on the user 330 (for example, a specific language or dialect) might 
also be selected by the controller. 

Upon selecting a video media object and an audio media object, such objects are then 
commonly decompressed 326 and 334, respectively, and provided to the composition generators 

25 3 28 and 336. The composition generator then generates output signals for presentation on the 



corresponding video and audio presentation devices 338 and 340. In short, the present invention 
enables content providers to target their advertising and programming to users by providing 
media objects which are selected either by the transmission side and/or the receiving side based 
upon a user's profile. 

5 One process preferably utilized by the aforementioned system to provide media object 

based targeted advertising and programming is illustrated in Figure 4. As shown, the process 
provides for the pushing of a media object based advertisement, which is targeted to specific 
users and does not rely upon the user selecting or specifying media objects. The process begins 
with determining profiles for the user audience (Block 402). It is to be appreciated that various 

10 methods may be used to determine a make-up of a given user audience, including, for example, 
using a sign-on and password to a system transmitting the program content. Any system or 
process for determining a user audience (including statistical processes) may be utilized. 
Preferably, the user profiles are determined at the transmitting system or by the media object 
creators such that media objects are pushed to users without requiring any user input or 

15 interaction. However, the user profiles, as discussed previously, may be determined at the 
receiving system or in combination with the receiving system. When the receiving system 
determines which media objects are to be presented to the user, multiple media objects are 
preferably pushed to the user and the receiving system merely selects one of the pushed media 
objects, based upon the user profile, and does not request media objects from the transmitting 

20 side. 

After the user profiles are determined, the process continues with the user selecting the 
desired programming (Block 404) to be received and presented. Preferably the user's choice of 
programming is communicated to the transmitting side (i.e., to the media object creators 108 and 
the transmitting system 102) by the receiving system 104. As is commonly known, many 
25 transmitting systems (especially cable systems and satellite systems) enable a user to select from 



numerous programming options at any instant. The present process preferably provides for the 
real-time communication of user programming choices back to the transmitting side such that 
targeted media objects may be pushed to the user for presentation during the selected 
programming signal. However, it is to be appreciated that the process of Figure 4 may also be 
5 accomplished in systems, which do not provide feedback of user programming choices to the 
transmitting side. In such systems, the receiving system 104 utilizes various stored profiling 
rules and protocols (which, for example, may be periodically downloaded and updated in the 
system or permanently built-in), which correspond to profiles pre-established at the transmitting 
side, to determine which media objects in a selected program to present to a user, based upon the 
10 user's profile. 

Additionally, it is to be appreciated, that the media objects selected for transmission to a 
user may be in conformance with a user profile, or may be items which attempt to steer a user 
towards a second profile. For example, a user profile indicating that a user prefers Coca-Cola ® 
might result in media objects associated with Pepsi® products being presented during the 

15 program, perhaps as an attempt to lure the user into trying Pepsi products. Further, as discussed 
above, such media objects may be provided as separate advertisements or in the program itself. 
For example, a program may contain a scene in which the actor drinks from a soft drink can. The 
present invention enables content creators to specify from which soft drink can the actor drinks 
by using media objects which are selected by the transmitting system or the receiving system 

20 based upon a user profile. For example, the actor may drink from a Pepsi can instead of a Coca- 
Cola can. Those skilled in the art appreciate that MPEG-4 and other object based formats enable 
the content creators, the transmitting system, and/or the receiving system to effortlessly and 
seamlessly replace a first media object of a Pepsi can with a second media object of a Coke can, 
such that the actor appears to drink from the desired product. 
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After the programming choice has been made by the user, the process preferably 
continues with pushing multi-object programming data to the user (Block 406). Preferably, as 
mentioned previously, the pushed objects are based upon profiles established on the transmitting 
side based upon the user viewing habits and programming choices. Additionally, for any given 
program, media object creators generally desire to create media objects for a given number of 
profiles (for example, Coca-Cola drinkers might be a first profile, while Pepsi drinkers are a 
second profile). Instead of creating a unique media object for each user's profile, the process 
preferably pushes media obj ects to specific users based upon user identifiers. Such identifiers are 
periodically loaded into the receiving system 104 and/or data storage device 1 14 such that when 
the receiving system 104 receives the multi-object programming data, the receiving system 104 is 
programmed to select a specific media object based upon the user identifier. It is to be 
appreciated, however, that the identification of users who are to receive specific media objects 
may also be determined and accomplished by various other commonly known data routing and 
distribution techniques, for example assigning unique nodes on a network to each user. The 
present invention is not limited to any specific embodiment, and as bi-directional data link 
throughput capacity increases, various other routing techniques may be utilized to push media 
object based targeted advertising and programming to users. 

Upon receiving the multi-object programming signal, the receiving system 104 then 
selects specific media objects for decompression, composition and presentation based upon the 
user's profile (Block 408). The selection of the media objects may be accomplished 
independently by the receiving system, in conjunction with the user profiling system, and/or in 
conjunction with the transmitting side. Additionally, the receiving system 104 maybe configured 
such that the media objects and/or profiles available are selected by the user via the user input 
device 110. Such user control features might prove popular, for example, when a user is 
watching a program, which for an adult profile contains adult content, but for an adolescent 
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profile does not contain the adult content. By merely selecting the appropriate profile (with 
passwords, voice recognition, or other features preventing unauthorized access to the adult 
content), the system 100 may be configured, for example, to allow parents to control the content 
of programs, and prevent adolescents from watching adult oriented content. 
5 At this point of the process, the media objects to be presented to the user have been 

selected. The process then continues with synchronizing the various objects and presenting the 
programming (Block 410). Since each program may contain hundreds of media objects, the 
process determines whether the selected program has been terminated (for example, by switching 
to another channel) (Block 412). If the programming has not been switched, the process ends and 

10 restarts with the selection of the new programming signal since a user's profile may vary 
depending upon the chosen programming. For example, a Kansas State fan watching a national 
championship football game between Kansas State University and Virginia Tech would probably 
be interested in receiving an audio broadcast (e.g., player, coach, in-stadium announcer, or radio 
broadcast), highlights, and other information which is oriented towards Kansas State, whereas the 

15 Virginia Tech fan would prefer to hear the audio broadcast, etc., which is oriented towards their 
team. By using user profiles and media object based content, each fan can receive the 
programming they desire. However, when watching the PGA championships, both users may 
have the same profile and thus do not need separate audio programming, etc. Thus, the system 
1 00 may also be configured to accommodate multiple profiles for each user, wherein the profiles 

20 are dependent upon the programming selected or other variables. 

Figure 5 illustrates a process flow utilized to pull media objects by a user. The pulling of 
content occurs when a user actively requests information from a provider. Commonly, such pulls 
occur, for example, by a user selecting a particular Web page on the Internet, selecting a 
particular data feed, etc. In addition to pulling specific programming, Web pages and other 

25 content, the present invention enables a user to pull specific media objects within a program. For 



example, a broadcast of a football game may provide users with the option of selecting a desired 
camera angle for a play (for example, the blimp view, the sideline view, the quarterback view, 
etc.) by providing multiple media objects within the programming signal. The present invention 
allows a user to select a media object(s) presenting the desired view and the corresponding audio 
5 (for example, the quarterbacks' s verbal commands). Alternatively, the user could pull up video 
stills or graphics/text enhancements for detailed statistics. Advertising can be included in such 
media objects (for example, the quarterback helmet cam might include advertising by Riddell®, 
the helmet manufacturer). 

As shown in Figure 5, the pull process begins with the transmitting side presenting 

10 programming which contains multiple media objects, which the user may select (Block 502). 
The receiving system 104 receives the programming signal and provides the user (when the user 
select option is enabled) with a menu of choices of media objects, view angles, etc. which can be 
selected via the user input device 110. However, the present invention is not limited to requiring 
a user to select a media object via menus and other traditional methods. Since each element of 

15 the programming signal is a distinct media object (for example, as provided in programming 
utilizing the MPEG-4 format), the user may simply select or designate the object and obtain the 
desired view or information. For example, during a broadcast of the Indy 500, a user could select 
a car on the screen and then receive the programming from the driver's viewpoint. Thus, the 
process of Figure 5 allows the user to determine which content (media objects) they desire to 

20 receive (Block 504). 

Upon the user's selection of a media object, the process continues with recording the user 
selection in the user profiling system (Block 506). The user's selections are utilized by the user 
profiling system to create a more complete profile of the user. For example, a user selecting a 
media object relating to hiking boots, is most likely interested in additional outdoor gear. Such 

25 information may be utilized to tailor advertising to such gear during breaks, for example, in 



football games. The process also provides information about the user's selections to the 
transmitting side (Block 507). The information provided may range from the highly specific to 
the non-specific and anonymous habits of a geographic region, as required by privacy constraints, 
data processing capabilities, and other factors. 

Upon receiving the user's selection of a media object, the process modifies the 
programming (Block 508). At this point the receiving system selects and displays the image from 
the quarterback cam or the particular racecar. Additionally, when a bi-directional 
communications link is provided between the transmitting side and the receiving system, media 
objects might actually be modified by the transmitting side prior to transmission. For example, 
the next commercial advertisement or programming segment may include images from the earlier 
selected view. 

Next, the process continues with the receiving system receiving the media objects, 
selecting the correct object, synchronizing the various objects and presenting the programming to 
the user (Block 510). Preferably, the selection of media objects by the user and the presentation 
of such objects occurs instantaneously from the user's perspective. Those skilled in the art 
appreciate the data processing speeds, buffers, transmission capabilities and other system 
capabilities that may be necessary to provide the before mentioned pull of media objects. The 
present invention may utilize any systems and processes, which provide such capabilities. 

When the media object being presented ends, the process continues with the regular 
programming being presented (Block 512). For example, after a play is viewed through the 
quarterback cam, the presentation returns to the sideline view or regular programming view and 
then, for example, allows the user to select another view or continue with the current view. The 
present invention may also be configured to continuously process a stream of media objects until 
a command is received from the user. During such a presentation, the system 100 continues to 
provide targeted advertising in the form of media objects, record user responses thereto, and 
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create/modify user profiles. In short, the process of Figure 5 enables a user to select and pull 
content, based upon their individual preferences, via the use of media objects. Such pulls may 
then be utilized to generate user profiles and target advertising to users based upon such profiles. 

While the present invention has been described in relation to a preferred system 
embodiment and various process flows, it will be appreciated by those skilled in the art that 
changes in the above descriptions or illustrations may be made with respect to form or detail 
without departing from the spirit and scope of the present invention. 
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